Finetune Translation (Generative Models)

Parameters

• storage type Determines where the finetuned model will be stored. Either in a folder in one of your projects / repositories (recommended), in a folder of your file system, or in a temporary folder.
• project folder The folder in a project / repository to store the finetuned model in.
• file folder The folder in your file system to store the finetuned model in.
• prompt prefix This prefix is used in front of each input to tell the finetuned model the task of this finetuning.
• input column The name of the attribute or column which should be used as input for the fine-tuning.
• target column The name of the attribute or column which should be used as the target for this fine-tuning. Since this is a translation task, the model will try to learn how to translate the values from the input column to those in the target column.
• max input tokens The maximum number of tokens allowed for the inputs. Longer sequences will be ignored.
• max target tokens The maximum number of tokens allowed for the target or output of the fine-tuned model. Longer sequences will be ignored.
• epochs The number of epochs for this fine-tuning.
• device Where the finetuning should take place. Either on a GPU, a CPU, or Apple’s MPS architecture. If set to Automatic, the training will prefer the GPU if available and will fall back to CPU otherwise.
• device indices If you have multiple GPUs and computation is set up to happen on GPUs you can specify which ones are used with this parameter. Counting of devices starts with 0. The default of “0” means that the first GPU device in the system will be used, a value of “1” would refer to the second and so on. You can utilize multiple GPUs by providing a comma-separated list of device indices. For example, you could use “0,1,2,3” on a machine with four GPUs if all four should be utilized. Please note that RapidMiner performs data-parallel computation which means that the model needs to be small enough to be completely loaded on each of your GPUs.
• finetuning mode Indicates if a full finetuning is performed or PEFT / LoRA which can dramatically accelerate the finetuning task.
• lora r The dimension of the low-rank matrices used by LoRA.
• lora alpha The scaling factor for the weight matrices used by LoRA.
• lora dropout The dropout probability of the LoRA layers.
• target modules mode If set to None, no specific definition is made for which modules (or layers) should be finetuned with PEFT / LoRA. This is the best setting for all the models which are natively supported by PEFT. If set to Automatic, we will extract the names of all linear layers automatically which is the recommended approach. And if set to Manual, you can specify a comma-separated list of target module names yourself.
• target modules Only shown if the target module mode is set to Manual. You can specify here a comma-separated list of target module names. Those modules would be finetuned with PEFT / LoRA then. You can see the structure of the model including the module names in the logs.
• quantization Quantization techniques reduce memory and computational costs by representing weights and activations with lower-precision data types like 8-bit or 4-bit integers.
• 16 bit precision Whether to use 16-bit (mixed) precision training (fp16) instead of 32-bit training.
• prep threads The number of parallel threads used for the data preprocessing.
• batch size The batch size for this fine-tuning. The number of GPUs x batch size x gradient accumulation steps should usually be a multiple of 8.
• gradient accumulation steps The gradient accumulation steps used for this fine-tuning. The number of GPUs x batch size x gradient accumulation steps should usually be a multiple of 8.
• train test ratio The ratio of rows which is used for testing the fine-tuned model.
• learning rate The learning rate for this fine-tuning.
• conda environment The conda environment used for this model task. Additional packages may be installed into this environment, please refer to the extension documentation for additional details on this and on version requirements for Python and some packages which have be present in this environment.